The current TP6 list is mostly sites with a single origin, and the highest number is (IIRC) 6.

We should expand the list to include a few high-eTLD+1-count sites, such as CNN.com (~14), espn.com (~14), wikia.com (22), expedia.com (8), nytimes.com (8), etc. Some of these we already look at for mobile-TP6. These are just ones I've used; we could use others - I just want to start the discussion.

This will improve our fission coverage, and also I think overall improve our testing. We have only a single news site currently, and not a high-usage one (yahoo news). This would add 3 news sites.

While useful for fission, I think the general TP6 list should be updated.

This isn't a mozperftest-specific issue (we don't use it for fission pageload tests either). Moving to Testing :: Raptor.

In bug 1677125 we're adding live site tests for desktop including CNN, ESPN, Expedia, NYTimes, and Wikia. Would this be sufficient or do we need to include recorded variants of these?

Will live sites be available in arewefastyet.com (or elsewhere)? I presume we'll test the same variants (firefox, webrender, fission, chrome, chromium)?

If we can get them (and get averages like we hope to get on arewefastyet), I think we can start with live sites. Or more to the point, we're interested in live sites as well as the recordings, and I think I'm ok with some sites only being live.

Is there a reason for their only being live? Cost to update?

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #4)

Will live sites be available in arewefastyet.com (or elsewhere)? I presume we'll test the same variants (firefox, webrender, fission, chrome, chromium)?

Yes, we should have the same variants. Details can be found in bug 1677125.

If we can get them (and get averages like we hope to get on arewefastyet), I think we can start with live sites. Or more to the point, we're interested in live sites as well as the recordings, and I think I'm ok with some sites only being live.

We have a pull request to add desktop live sites to AWFY that's just waiting on having some data to test with.

Is there a reason for their only being live? Cost to update?

Adding live sites is certainly the lowest cost, and something we were already working on which is why I asked if this would be sufficient. Recording sites requires additional work, especially as some sites can be problematic, but if this is something we need for Fission then adding new recordings can be prioritised.

Yes, we should add recordings as soon as we can. Moving this to Fission M7a where other test enablement is also being tracked.

Can we confirm the pages we want to add to the recorded suite? Based on the suggestions in comment 0, these sites exist in our live suite. Do we just want to record these, or do we need to make changes?

• CNN: https://cnn.com/ampstories/us/why-hurricane-michael-is-a-monster-unlike-any-other
• Wikia: https://marvel.wikia.com/wiki/Black_Panther
• Expedia: https://expedia.com/Hotel-Search?destination=New+York%2C+New+York&latLong=40.756680%2C-73.986470&regionId=178293&startDate=&endDate=&rooms=1&_xpid=11905%7C1&adults=2
• NYTimes: https://www.nytimes.com/2020/02/19/opinion/surprise-medical-bill.html

We don't currently test ESPN on desktop or live, but we do have the following recorded page load test on mobile, do we want to add this to our live/recorded desktop suites?

• ESPN: http://www.espn.com/nba/story/_/page/allstarweekend25788027/the-comparison-lebron-james-michael-jordan-their-own-words

As this will increase the total number of sites we test, are there any tests that we can consider removing from our current suites?

Those sites look good.

From the current list, we can probably drop Yahoo News. I think we could also drop Apple. Also: is the Facebook redesign the default now? More to the point, is the old version still in use?

While I'm tempted to remove one of gdocs/gslides/gsheets, since they're highly correlated, they're also important sites. @acreskey, @denispal, thoughts?

I'm very much in favor of adding sites with more origins.
Particularly for live sites where they stress TLS establishment which can be very expensive for new users.

Where can I find a list of the sites that we're currently using?

https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.ini

Or look at https://arewefastyet.com

I would consider also dropping https://yandex.ru/search/?text=barack%20obama&lr=10115 - I've personally found it to be too noisy to be very useful.

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #8)

While I'm tempted to remove one of gdocs/gslides/gsheets, since they're highly correlated, they're also important sites. @acreskey, @denispal, thoughts?

That makes sense. Maybe gslides ?

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #8)

Those sites look good.

From the current list, we can probably drop Yahoo News. I think we could also drop Apple. Also: is the Facebook redesign the default now? More to the point, is the old version still in use?

We're currently running both facebook and facebook-redesign. We will soon drop "-redesign" from the latter and remove the legacy Facebook test.

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #9)

I'm very much in favor of adding sites with more origins.
Particularly for live sites where they stress TLS establishment which can be very expensive for new users.

Where can I find a list of the sites that we're currently using?

https://firefox-source-docs.mozilla.org/testing/perfdocs/raptor.html

(In reply to Dave Hunt [:davehunt] [he/him] ⌚GMT from comment #13)

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #8)

Those sites look good.

From the current list, we can probably drop Yahoo News. I think we could also drop Apple. Also: is the Facebook redesign the default now? More to the point, is the old version still in use?

We're currently running both facebook and facebook-redesign. We will soon drop "-redesign" from the latter and remove the legacy Facebook test.

This will be taken care of in bug 1699764.

The sites proposed here look fine to me, but I'll just defer to jesup for this. If we want more options, we can also add:

https://dlisted.com ~11
http://www.justjared.com ~20
https://www.tmz.com ~7

and many other gossip related sites.

So I would add https://www.buzzfeed.com to the original list (I know of TMZ, I haven't even heard of the others - but I'm not into gossip). From experience, this is a big, complex, ad-heavy page that loads quickly, which makes it interesting. 8 eTLDs

Longer-term we should consider adding more non-NA/Europe sites - asia, india, etc.

To summarise, we're going to add the following sites (with recordings) to our desktop suite:

• cnn
• wikia
• expedia
• nytimes
• espn
• buzzfeed

We're also going to remove the following from the our desktop suite:

• apple
• yahoo-news
• facebook (to be replaced by facebook-redesign, see bug 1699764)
• google-slides
• yandex

:bebe can you coordindate this?

I've just noticed that google-slides appears on our list of high value tests (last updated in June). :sparky do you think we should keep this test? Is there another we could remove instead?

(In reply to Dave Hunt [:davehunt] [he/him] ⌚GMT from comment #19)

I've just noticed that google-slides appears on our list of high value tests (last updated in June). :sparky do you think we should keep this test? Is there another we could remove instead?

I think we should remove one of the other google-* tests. The google-slides test still shows up as a high-value test. Here's are the high-value tests since September 2020 for raptor and browsertime:

Chosen tests: ['raptor-tp6-youtube-firefox-cold', 'google-slides', 'raptor-tp6-netflix-firefox-cold', 'raptor-tp6-tumblr-firefox-cold', 'raptor-tp6-wikipedia-firefox-cold', 'raptor-stylebench-firefox', 'raptor-tp6-bing-firefox-cold', 'youtube', 'raptor-tp6-cnn-ampstories-firefox-cold', 'raptor-motionmark-htmlsuite-firefox', 'raptor-assorted-dom-firefox', 'linkedin', 'raptor-tp6-google-mail-firefox-cold', 'raptor-tp6-slides-firefox-cold', 'raptor-tp6-linkedin-firefox-cold', 'reddit']

So gdocs or gsheets would be ok to get rid of and replace with one of the new ones in this list (I think linkedin, or reddit would be the best one to add).

(In reply to Greg Mierzwinski [:sparky] from comment #20)

So gdocs or gsheets would be ok to get rid of and replace with one of the new ones in this list (I think linkedin, or reddit would be the best one to add).

:jesup do you have a preference here? Also I noticed that https://cnn.com/ampstories/us/why-hurricane-michael-is-a-monster-unlike-any-other appears to be optimised for mobile, should we use a desktop equivalent instead?

Yeah, we shouldn't use the amp version. I couldn't easily find that article, so I selected another random article off CNN:
https://www.cnn.com/2021/03/22/weather/climate-change-warm-waters-lake-michigan/index.html

Of gsuite, I think the best one to remove would be the gsheets test. Keep gdocs and gslides.

https://hg.mozilla.org/mozilla-central/rev/f2bca9b2da98

reopen this to remove the other sites
and add live sites variant for the added sites

https://hg.mozilla.org/mozilla-central/rev/704199bfdb3e